SherbyNet - /page/wikibook/entry.mu

<!DOCTYPE html>

<title>Positive-definite kernel</title>

</head>

Positive-definite kernel

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">"Kernel function" redirects here; not to be confused with <a href="Integral_kernel" class="mw-redirect" title="Integral kernel">Integral kernel</a>.</div>

In <a href="Operator_theory" title="Operator theory">operator theory</a>, a branch of mathematics, a positive-definite kernel is a generalization of a <a href="Positive-definite_function" title="Positive-definite function">positive-definite function</a> or a <a href="Positive-definite_matrix" class="mw-redirect" title="Positive-definite matrix">positive-definite matrix</a>. It was first introduced by <a href="James_Mercer_(mathematician)" title="James Mercer (mathematician)">James Mercer</a> in the early 20th century, in the context of solving <a href="Integral_equation" title="Integral equation">integral operator equations</a>. Since then, positive-definite functions and their various analogues and generalizations have arisen in diverse parts of mathematics. They occur naturally in <a href="Fourier_analysis" title="Fourier analysis">Fourier analysis</a>, <a href="Probability_theory" title="Probability theory">probability theory</a>, <a href="Operator_theory" title="Operator theory">operator theory</a>, <a href="Complex_analysis" title="Complex analysis">complex function-theory</a>, <a href="Moment_problem" title="Moment problem">moment problems</a>, <a href="Integral_equation" title="Integral equation">integral equations</a>, <a href="Boundary_value_problem" title="Boundary value problem">boundary-value problems</a> for <a href="Partial_differential_equation" title="Partial differential equation">partial differential equations</a>, <a href="Machine_learning" title="Machine learning">machine learning</a>, <a href="Embedding_problem" title="Embedding problem">embedding problem</a>, <a href="Information_theory" title="Information theory">information theory</a>, and other areas.

<div class="mw-heading mw-heading2"><h2 id="Definition">Definition</h2></div>

Let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {X}}}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./8c7e5461c5286852df4ef652fca7e4b0b63030e9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.875ex; height:2.176ex;" alt="{\displaystyle {\mathcal {X}}}" loading="lazy"> be a nonempty set, sometimes referred to as the index set. A <a href="Symmetric_function" title="Symmetric function">symmetric function</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K:{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {R} }">

</mrow>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K:{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {R} }</annotation>

</semantics>

</math><img src="./7e099af6495d5cb2bf18d85a15ec491beb770394.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:15.887ex; height:2.176ex;" alt="{\displaystyle K:{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {R} }" loading="lazy"> is called a positive-definite (p.d.) kernel on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {X}}}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./8c7e5461c5286852df4ef652fca7e4b0b63030e9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.875ex; height:2.176ex;" alt="{\displaystyle {\mathcal {X}}}" loading="lazy"> if

/* start https://en.wikipedia.org/ */

.mw-parser-output table.numblk{border-collapse:collapse;border:none;margin-top:0;margin-right:0;margin-bottom:0}.mw-parser-output table.numblk>tbody>tr>td{vertical-align:middle;padding:0}.mw-parser-output table.numblk>tbody>tr>td:nth-child(2){width:99%}.mw-parser-output table.numblk>tbody>tr>td:nth-child(2)>table{border-collapse:collapse;margin:0;border:none;width:100%}.mw-parser-output table.numblk>tbody>tr>td:nth-child(2)>table>tbody>tr:first-child>td:first-child,.mw-parser-output table.numblk>tbody>tr>td:nth-child(2)>table>tbody>tr:first-child>td:last-child{padding:0 0.4ex}.mw-parser-output table.numblk>tbody>tr>td:nth-child(2)>table>tbody>tr:first-child>td:nth-child(2){width:100%;padding:0}.mw-parser-output table.numblk>tbody>tr>td:nth-child(2)>table>tbody>tr:last-child>td{padding:0}.mw-parser-output table.numblk>tbody>tr>td:last-child{font-weight:bold}.mw-parser-output table.numblk.numblk-raw-n>tbody>tr>td:last-child{font-weight:unset}.mw-parser-output table.numblk>tbody>tr>td:last-child::before{content:"("}.mw-parser-output table.numblk>tbody>tr>td:last-child::after{content:")"}.mw-parser-output table.numblk.numblk-raw-n>tbody>tr>td:last-child::before,.mw-parser-output table.numblk.numblk-raw-n>tbody>tr>td:last-child::after{content:none}.mw-parser-output table.numblk>tbody>tr>td{border:none}.mw-parser-output table.numblk.numblk-border>tbody>tr>td{border:thin solid}.mw-parser-output table.numblk>tbody>tr>td:nth-child(2)>table>tbody>tr:first-child>td{border:none}.mw-parser-output table.numblk.numblk-border>tbody>tr>td:nth-child(2)>table>tbody>tr:first-child>td{border:thin solid}.mw-parser-output table.numblk>tbody>tr>td:nth-child(2)>table>tbody>tr:last-child>td{border-left:none;border-right:none;border-bottom:none}.mw-parser-output table.numblk.numblk-border>tbody>tr>td:nth-child(2)>table>tbody>tr:last-child>td{border-left:thin solid;border-right:thin solid;border-bottom:thin solid}.mw-parser-output table.numblk:target{color:var(--color-base,#202122);background-color:#cfe8fd}@media screen{html.skin-theme-clientpref-night .mw-parser-output table.numblk:target{color:var(--color-base,#eaecf0);background-color:#301702}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output table.numblk:target{color:var(--color-base,#eaecf0);background-color:#301702}}

/* end https://en.wikipedia.org/ */

</style><table role="presentation" class="numblk" style="margin-left: 0em;"><tbody><tr><td class="nowrap"><math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \sum _{i=1}^{n}\sum _{j=1}^{n}c_{i}c_{j}K(x_{i},x_{j})\geq 0}">

</mrow>

</mrow>

</munderover>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sum _{i=1}^{n}\sum _{j=1}^{n}c_{i}c_{j}K(x_{i},x_{j})\geq 0}</annotation>

</semantics>

</math></td> <td></td> <td class="nowrap">1.1</td></tr></tbody></table>

holds for all <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./17bb9dc93c447ce33bb7fbcbc2e147823ce03dc0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.826ex; height:2.509ex;" alt="{\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}" loading="lazy">, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\in \mathbb {N} ,c_{1},\dots ,c_{n}\in \mathbb {R} }">

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\in \mathbb {N} ,c_{1},\dots ,c_{n}\in \mathbb {R} }</annotation>

</semantics>

</math><img src="./1b8bea740b4b06aa3253a2896852896ef28a0d7b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:20.931ex; height:2.509ex;" alt="{\displaystyle n\in \mathbb {N} ,c_{1},\dots ,c_{n}\in \mathbb {R} }" loading="lazy">.

In probability theory, a distinction is sometimes made between positive-definite kernels, for which equality in (1.1) implies <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle c_{i}=0\;(\forall i)}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle c_{i}=0\;(\forall i)}</annotation>

</semantics>

</math><img src="./bf0fd6c8895a2b81903386f37f87dee6c39b38bc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:10.617ex; height:2.843ex;" alt="{\displaystyle c_{i}=0\;(\forall i)}" loading="lazy">, and positive semi-definite (p.s.d.) kernels, which do not impose this condition. Note that this is equivalent to requiring that every finite matrix constructed by pairwise evaluation, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {K} _{ij}=K(x_{i},x_{j})}">

<msub>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbf {K} _{ij}=K(x_{i},x_{j})}</annotation>

</semantics>

</math><img src="./49b9d95083e200f397670e2ce290225e8f5d846e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:15.947ex; height:3.009ex;" alt="{\displaystyle \mathbf {K} _{ij}=K(x_{i},x_{j})}" loading="lazy">, has either entirely positive (p.d.) or nonnegative (p.s.d.) <a href="Eigenvalues_and_eigenvectors" title="Eigenvalues and eigenvectors">eigenvalues</a>.

In mathematical literature, kernels are usually complex-valued functions. That is, a complex-valued function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K:{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {C} }">

</mrow>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K:{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {C} }</annotation>

</semantics>

</math><img src="./667c1b939fcacc6e6db92ae9bc31acfd98d66db0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:15.887ex; height:2.176ex;" alt="{\displaystyle K:{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {C} }" loading="lazy"> is called a Hermitian kernel if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(x,y)={\overline {K(y,x)}}}">

<mover>

<mrow>

</mrow>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)={\overline {K(y,x)}}}</annotation>

</semantics>

</math><img src="./37c3176d4092c55fda1dd599799b6e7fba5ead1d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:18.002ex; height:3.676ex;" alt="{\displaystyle K(x,y)={\overline {K(y,x)}}}" loading="lazy"> and positive definite if for every finite set of points <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./17bb9dc93c447ce33bb7fbcbc2e147823ce03dc0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.826ex; height:2.509ex;" alt="{\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}" loading="lazy"> and any complex numbers <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \xi _{1},\dots ,\xi _{n}\in \mathbb {C} }">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \xi _{1},\dots ,\xi _{n}\in \mathbb {C} }</annotation>

</semantics>

</math><img src="./8396d2f9363d02da78fc8a1e26d1a78e4b976c39.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.007ex; height:2.509ex;" alt="{\displaystyle \xi _{1},\dots ,\xi _{n}\in \mathbb {C} }" loading="lazy">,

</mrow>

</mrow>

</munderover>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sum _{i=1}^{n}\sum _{j=1}^{n}\xi _{i}{\overline {\xi }}_{j}K(x_{i},x_{j})\geq 0}</annotation>

</semantics>

</math><img src="./2397d0f98353cb945b676d3534a651d48ebdea46.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.338ex; width:24.904ex; height:7.176ex;" alt="{\displaystyle \sum _{i=1}^{n}\sum _{j=1}^{n}\xi _{i}{\overline {\xi }}_{j}K(x_{i},x_{j})\geq 0}" loading="lazy"></dd></dl>

where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\overline {\xi }}_{j}}">

<msub>

<mover>

</mover>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\overline {\xi }}_{j}}</annotation>

</semantics>

</math><img src="./e2bc8f5a3958cc731963f1ee8c4af02d9887d1d4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.171ex; width:2.062ex; height:3.843ex;" alt="{\displaystyle {\overline {\xi }}_{j}}" loading="lazy"> denotes the <a href="Complex_conjugate" title="Complex conjugate">complex conjugate</a>.<a href="#cite_note-1">[1]</a> In the rest of this article we assume real-valued functions, which is the common practice in applications of p.d. kernels.

<div class="mw-heading mw-heading3"><h3 id="Some_general_properties">Some general properties</h3></div>

<ul><li>For a family of p.d. kernels <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (K_{i})_{i\in \mathbb {N} },\ \ K_{i}:{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {R} }">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (K_{i})_{i\in \mathbb {N} },\ \ K_{i}:{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {R} }</annotation>

</semantics>

</math><img src="./8b550066b4f6187d8ed1a6fff31039d5286fe3a6.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:26.453ex; height:2.843ex;" alt="{\displaystyle (K_{i})_{i\in \mathbb {N} },\ \ K_{i}:{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {R} }" loading="lazy">

<ul><li>The conical sum <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \sum _{i=1}^{n}\lambda _{i}K_{i}}">

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sum _{i=1}^{n}\lambda _{i}K_{i}}</annotation>

</semantics>

</math><img src="./4c5fd7b55d8120cfb8c04d014dcde873b66d6147.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.005ex; width:8.67ex; height:6.843ex;" alt="{\displaystyle \sum _{i=1}^{n}\lambda _{i}K_{i}}" loading="lazy"> is p.d., given <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lambda _{1},\dots ,\lambda _{n}\geq 0}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \lambda _{1},\dots ,\lambda _{n}\geq 0}</annotation>

</semantics>

</math><img src="./970ecd7d0d73571335b11afc8da736484e85cee5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.422ex; height:2.509ex;" alt="{\displaystyle \lambda _{1},\dots ,\lambda _{n}\geq 0}" loading="lazy"></li>

<li>The product <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K_{1}^{a_{1}}\dots K_{n}^{a_{n}}}">

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K_{1}^{a_{1}}\dots K_{n}^{a_{n}}}</annotation>

</semantics>

</math><img src="./cdb0a9fcc772ff68f22c8b82c90792ab6bcb333c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:11.685ex; height:3.176ex;" alt="{\displaystyle K_{1}^{a_{1}}\dots K_{n}^{a_{n}}}" loading="lazy"> is p.d., given <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle a_{1},\dots ,a_{n}\in \mathbb {N} }">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle a_{1},\dots ,a_{n}\in \mathbb {N} }</annotation>

</semantics>

</math><img src="./8252faf9b91164e2dda528522fe89146e7935d2b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.429ex; height:2.509ex;" alt="{\displaystyle a_{1},\dots ,a_{n}\in \mathbb {N} }" loading="lazy"></li>

<li>The limit <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K=\lim _{n\to \infty }K_{n}}">

</mrow>

</munder>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K=\lim _{n\to \infty }K_{n}}</annotation>

</semantics>

</math><img src="./f021facaab413d4dca9b380030f9d57308542b00.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; width:13.016ex; height:3.676ex;" alt="{\displaystyle K=\lim _{n\to \infty }K_{n}}" loading="lazy"> is p.d. if the limit exists.</li></ul></li>

<msub>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle ({\mathcal {X}}_{i})_{i=1}^{n}}</annotation>

</semantics>

</math><img src="./985cc7f5e996b14ab1664caa15a3a1a49222e35f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:7.166ex; height:3.009ex;" alt="{\displaystyle ({\mathcal {X}}_{i})_{i=1}^{n}}" loading="lazy"> is a sequence of sets, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (K_{i})_{i=1}^{n},\ \ K_{i}:{\mathcal {X}}_{i}\times {\mathcal {X}}_{i}\to \mathbb {R} }">

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</msubsup>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (K_{i})_{i=1}^{n},\ \ K_{i}:{\mathcal {X}}_{i}\times {\mathcal {X}}_{i}\to \mathbb {R} }</annotation>

</semantics>

</math><img src="./e08012440741b8eefc0af05988f42bd4edb6f794.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:27.434ex; height:3.009ex;" alt="{\displaystyle (K_{i})_{i=1}^{n},\ \ K_{i}:{\mathcal {X}}_{i}\times {\mathcal {X}}_{i}\to \mathbb {R} }" loading="lazy"> a sequence of p.d. kernels, then both <math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K((x_{1},\dots ,x_{n}),(y_{1},\dots ,y_{n}))=\prod _{i=1}^{n}K_{i}(x_{i},y_{i})}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K((x_{1},\dots ,x_{n}),(y_{1},\dots ,y_{n}))=\prod _{i=1}^{n}K_{i}(x_{i},y_{i})}</annotation>

</semantics>

</math> and <math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K((x_{1},\dots ,x_{n}),(y_{1},\dots ,y_{n}))=\sum _{i=1}^{n}K_{i}(x_{i},y_{i})}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K((x_{1},\dots ,x_{n}),(y_{1},\dots ,y_{n}))=\sum _{i=1}^{n}K_{i}(x_{i},y_{i})}</annotation>

</semantics>

</math> are p.d. kernels on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {X}}={\mathcal {X}}_{1}\times \dots \times {\mathcal {X}}_{n}}">

</mrow>

<msub>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}={\mathcal {X}}_{1}\times \dots \times {\mathcal {X}}_{n}}</annotation>

</semantics>

</math><img src="./af6d6c3d9a353c08f81cb28ee4ce3eaedeca4fbb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:18.965ex; height:2.509ex;" alt="{\displaystyle {\mathcal {X}}={\mathcal {X}}_{1}\times \dots \times {\mathcal {X}}_{n}}" loading="lazy">.</li>

<msub>

</mrow>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}_{0}\subset {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./ca21dab37b8232a13bd3d3e5c484f518e0a13ddc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:7.685ex; height:2.509ex;" alt="{\displaystyle {\mathcal {X}}_{0}\subset {\mathcal {X}}}" loading="lazy">. Then the restriction <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K_{0}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K_{0}}</annotation>

</semantics>

</math><img src="./44b0af6cafb690d3dbb0f3f30a032631338dc476.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.027ex; height:2.509ex;" alt="{\displaystyle K_{0}}" loading="lazy"> of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy"> to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {X}}_{0}\times {\mathcal {X}}_{0}}">

<msub>

</mrow>

</mrow>

</msub>

<msub>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}_{0}\times {\mathcal {X}}_{0}}</annotation>

</semantics>

</math><img src="./57807015cb10292bb6c84550b9e24be84cf4cf3c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:8.263ex; height:2.509ex;" alt="{\displaystyle {\mathcal {X}}_{0}\times {\mathcal {X}}_{0}}" loading="lazy"> is also a p.d. kernel.</li></ul>

<div class="mw-heading mw-heading3"><h3 id="Examples_of_p.d._kernels">Examples of p.d. kernels</h3></div>

<ul><li>Common examples of p.d. kernels defined on Euclidean space <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbb {R} ^{d}}">

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbb {R} ^{d}}</annotation>

</semantics>

</math><img src="./a713426956296f1668fce772df3c60b9dde8a685.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.77ex; height:2.676ex;" alt="{\displaystyle \mathbb {R} ^{d}}" loading="lazy"> include:

<ul><li>Linear kernel: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=\mathbf {x} ^{T}\mathbf {y} ,\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d}}">

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(\mathbf {x} ,\mathbf {y} )=\mathbf {x} ^{T}\mathbf {y} ,\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d}}</annotation>

</semantics>

</math><img src="./32dbbcb898d2fac3cf20c00c5ef2ee66109c9f17.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:27.864ex; height:3.176ex;" alt="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=\mathbf {x} ^{T}\mathbf {y} ,\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d}}" loading="lazy">.</li>

<li><a href="Polynomial_kernel" title="Polynomial kernel">Polynomial kernel</a>: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=(\mathbf {x} ^{T}\mathbf {y} +r)^{n},\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d},r\geq 0,n\geq 1}">

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

<msup>

</mrow>

</msup>

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(\mathbf {x} ,\mathbf {y} )=(\mathbf {x} ^{T}\mathbf {y} +r)^{n},\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d},r\geq 0,n\geq 1}</annotation>

</semantics>

</math><img src="./5664c78b4d024409d88250ec34de91038aa98431.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:47.814ex; height:3.176ex;" alt="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=(\mathbf {x} ^{T}\mathbf {y} +r)^{n},\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d},r\geq 0,n\geq 1}" loading="lazy">.</li>

<li><a href="Gaussian_kernel" class="mw-redirect" title="Gaussian kernel">Gaussian kernel</a> (<a href="Radial_basis_function_kernel" title="Radial basis function kernel">RBF kernel</a>): <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=e^{-{\frac {\|\mathbf {x} -\mathbf {y} \|^{2}}{2\sigma ^{2}}}},\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d},\sigma >0}">

</mrow>

</mrow>

<msup>

<mfrac>

<mrow>

</mrow>

</mrow>

<msup>

</mrow>

</msup>

</mrow>

<mrow>

<msup>

</mrow>

</msup>

</mrow>

</mfrac>

</mrow>

</msup>

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(\mathbf {x} ,\mathbf {y} )=e^{-{\frac {\|\mathbf {x} -\mathbf {y} \|^{2}}{2\sigma ^{2}}}},\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d},\sigma >0}</annotation>

</semantics>

</math><img src="./905a11cbc2df2b98810f7bb7cd9566023906e90d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:38.501ex; height:5.176ex;" alt="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=e^{-{\frac {\|\mathbf {x} -\mathbf {y} \|^{2}}{2\sigma ^{2}}}},\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d},\sigma >0}" loading="lazy">.</li>

<li>Laplacian kernel: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=e^{-\alpha \|\mathbf {x} -\mathbf {y} \|},\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d},\alpha >0}">

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</mrow>

</msup>

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(\mathbf {x} ,\mathbf {y} )=e^{-\alpha \|\mathbf {x} -\mathbf {y} \|},\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d},\alpha >0}</annotation>

</semantics>

</math><img src="./478701c819a08ebe7996794641b03000cb3be784.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:38.999ex; height:3.343ex;" alt="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=e^{-\alpha \|\mathbf {x} -\mathbf {y} \|},\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d},\alpha >0}" loading="lazy">.</li>

<li>Abel kernel: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(x,y)=e^{-\alpha |x-y|},\quad x,y\in \mathbb {R} ,\alpha >0}">

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)=e^{-\alpha |x-y|},\quad x,y\in \mathbb {R} ,\alpha >0}</annotation>

</semantics>

</math><img src="./9f29599007b1a363f378646165215be3f8e19ea6.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:36.266ex; height:3.343ex;" alt="{\displaystyle K(x,y)=e^{-\alpha |x-y|},\quad x,y\in \mathbb {R} ,\alpha >0}" loading="lazy">.</li>

<li>Kernel generating <a href="Sobolev_space" title="Sobolev space">Sobolev spaces</a> <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle W_{2}^{k}(\mathbb {R} ^{d})}">

</mrow>

</mrow>

</msubsup>

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle W_{2}^{k}(\mathbb {R} ^{d})}</annotation>

</semantics>

</math><img src="./900288b2517470111aa59fa0d71fd18cd8508ed9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:8.176ex; height:3.343ex;" alt="{\displaystyle W_{2}^{k}(\mathbb {R} ^{d})}" loading="lazy">: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(x,y)=\|x-y\|_{2}^{k-{\frac {d}{2}}}B_{k-{\frac {d}{2}}}(\|x-y\|_{2})}">

</mrow>

<mfrac>

</mfrac>

</mrow>

</msubsup>

<msub>

<mfrac>

</mfrac>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)=\|x-y\|_{2}^{k-{\frac {d}{2}}}B_{k-{\frac {d}{2}}}(\|x-y\|_{2})}</annotation>

</semantics>

</math><img src="./6239bd103d297ebc7951c53fe4c1132a847e0f94.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.171ex; width:38.224ex; height:6.009ex;" alt="{\displaystyle K(x,y)=\|x-y\|_{2}^{k-{\frac {d}{2}}}B_{k-{\frac {d}{2}}}(\|x-y\|_{2})}" loading="lazy">, where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle B_{\nu }}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle B_{\nu }}</annotation>

</semantics>

</math><img src="./167d5aef272a5cefef840078bcfa940682a75c24.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.868ex; height:2.509ex;" alt="{\displaystyle B_{\nu }}" loading="lazy"> is the <a href="Bessel_function#Hankel_functions" title="Bessel function">Bessel function of the third kind</a>.</li>

<li>Kernel generating Paley–Wiener space: <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(x,y)=\operatorname {sinc} (\alpha (x-y)),\quad x,y\in \mathbb {R} ,\alpha >0}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)=\operatorname {sinc} (\alpha (x-y)),\quad x,y\in \mathbb {R} ,\alpha >0}</annotation>

</semantics>

</math><img src="./541970ed409fb5ab1b00c5eae41d065bd4748f0c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:42.989ex; height:2.843ex;" alt="{\displaystyle K(x,y)=\operatorname {sinc} (\alpha (x-y)),\quad x,y\in \mathbb {R} ,\alpha >0}" loading="lazy">.</li></ul></li>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H}</annotation>

</semantics>

</math><img src="./75a9edddcca2f782014371f75dca39d7e13a9c1b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.064ex; height:2.176ex;" alt="{\displaystyle H}" loading="lazy"> is a <a href="Hilbert_space" title="Hilbert space">Hilbert space</a>, then its corresponding inner product <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (\cdot ,\cdot )_{H}:H\times H\to \mathbb {R} }">

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (\cdot ,\cdot )_{H}:H\times H\to \mathbb {R} }</annotation>

</semantics>

</math><img src="./c8a0fc86598149cb0de1720b57aa8aedf5cca412.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:20.025ex; height:2.843ex;" alt="{\displaystyle (\cdot ,\cdot )_{H}:H\times H\to \mathbb {R} }" loading="lazy"> is a p.d. kernel. Indeed, we have <math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \sum _{i,j=1}^{n}c_{i}c_{j}(x_{i},x_{j})_{H}=\left(\sum _{i=1}^{n}c_{i}x_{i},\sum _{j=1}^{n}c_{j}x_{j}\right)_{H}=\left\|\sum _{i=1}^{n}c_{i}x_{i}\right\|_{H}^{2}\geq 0}">

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

<mrow>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msub>

<mrow>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sum _{i,j=1}^{n}c_{i}c_{j}(x_{i},x_{j})_{H}=\left(\sum _{i=1}^{n}c_{i}x_{i},\sum _{j=1}^{n}c_{j}x_{j}\right)_{H}=\left\|\sum _{i=1}^{n}c_{i}x_{i}\right\|_{H}^{2}\geq 0}</annotation>

</semantics>

</math></li>

<li>Kernels defined on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbb {R} _{+}^{d}}">

</mrow>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbb {R} _{+}^{d}}</annotation>

</semantics>

</math><img src="./211d99435f22086c918480751cde3b7b8990ebdd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:3.189ex; height:3.343ex;" alt="{\displaystyle \mathbb {R} _{+}^{d}}" loading="lazy"> and histograms: Histograms are frequently encountered in applications of real-life problems. Most observations are usually available under the form of nonnegative vectors of counts, which, if normalized, yield histograms of frequencies. It has been shown <a href="#cite_note-2">[2]</a> that the following family of squared metrics, respectively Jensen divergence, the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \chi }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \chi }</annotation>

</semantics>

</math><img src="./656111758322ace96d80a9371771aa6d3de25437.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.455ex; height:2.009ex;" alt="{\displaystyle \chi }" loading="lazy">-square, Total Variation, and two variations of the Hellinger distance:<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \psi _{JD}=H\left({\frac {\theta +\theta '}{2}}\right)-{\frac {H(\theta )+H(\theta ')}{2}},}">

<msub>

</mrow>

</msub>

<mrow>

<mfrac>

<mrow>

<msup>

</msup>

</mrow>

</mfrac>

</mrow>

</mrow>

<mfrac>

<mrow>

<msup>

</msup>

</mrow>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \psi _{JD}=H\left({\frac {\theta +\theta '}{2}}\right)-{\frac {H(\theta )+H(\theta ')}{2}},}</annotation>

</semantics>

</math><math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \psi _{\chi ^{2}}=\sum _{i}{\frac {(\theta _{i}-\theta _{i}')^{2}}{\theta _{i}+\theta _{i}'}},\quad \psi _{TV}=\sum _{i}\left|\theta _{i}-\theta _{i}'\right|,}">

<msub>

<msup>

</mrow>

</msup>

</mrow>

</msub>

</mrow>

</munder>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

<msup>

</mrow>

</msup>

</mrow>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mrow>

</mfrac>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<mrow>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</msubsup>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \psi _{\chi ^{2}}=\sum _{i}{\frac {(\theta _{i}-\theta _{i}')^{2}}{\theta _{i}+\theta _{i}'}},\quad \psi _{TV}=\sum _{i}\left|\theta _{i}-\theta _{i}'\right|,}</annotation>

</semantics>

</math><math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \psi _{H_{1}}=\sum _{i}\left|{\sqrt {\theta _{i}}}-{\sqrt {\theta _{i}'}}\right|,\psi _{H_{2}}=\sum _{i}\left|{\sqrt {\theta _{i}}}-{\sqrt {\theta _{i}'}}\right|^{2},}">

<msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

</mrow>

</munder>

<mrow>

<mrow>

<msqrt>

<msub>

</mrow>

</msub>

</msqrt>

</mrow>

<msqrt>

</mrow>

</msubsup>

</msqrt>

</mrow>

</mrow>

<msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

</mrow>

</munder>

<msup>

<mrow>

<mrow>

<msqrt>

<msub>

</mrow>

</msub>

</msqrt>

</mrow>

<msqrt>

</mrow>

</msubsup>

</msqrt>

</mrow>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \psi _{H_{1}}=\sum _{i}\left|{\sqrt {\theta _{i}}}-{\sqrt {\theta _{i}'}}\right|,\psi _{H_{2}}=\sum _{i}\left|{\sqrt {\theta _{i}}}-{\sqrt {\theta _{i}'}}\right|^{2},}</annotation>

</semantics>

</math>can be used to define p.d. kernels using the following formula<math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(\theta ,\theta ')=e^{-\alpha \psi (\theta ,\theta ')},\alpha >0.}">

<msup>

</msup>

<msup>

<msup>

</msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(\theta ,\theta ')=e^{-\alpha \psi (\theta ,\theta ')},\alpha >0.}</annotation>

</semantics>

</math></li></ul>

<div class="mw-heading mw-heading3"><h3 id="Examples_of_other_kernels">Examples of other kernels</h3></div>

The sigmoid kernel, or hyperbolic tangent kernel, is defined as <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=\tanh(\gamma \mathbf {x} ^{T}\mathbf {y} +r),\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d}}">

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(\mathbf {x} ,\mathbf {y} )=\tanh(\gamma \mathbf {x} ^{T}\mathbf {y} +r),\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d}}</annotation>

</semantics>

</math><img src="./5053214b41ca445bdfebd67d08053bdd8bb7aee7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:39.477ex; height:3.176ex;" alt="{\displaystyle K(\mathbf {x} ,\mathbf {y} )=\tanh(\gamma \mathbf {x} ^{T}\mathbf {y} +r),\quad \mathbf {x} ,\mathbf {y} \in \mathbb {R} ^{d}}" loading="lazy"> where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \gamma ,r}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \gamma ,r}</annotation>

</semantics>

</math><img src="./e189187a25f430de5142f061c3d3917cb4e7081e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:3.345ex; height:2.176ex;" alt="{\displaystyle \gamma ,r}" loading="lazy"> are real parameters. The kernel is not PD, but has been sometimes used for kernel algorithms.<a href="#cite_note-3">[3]</a>

<div class="mw-heading mw-heading2"><h2 id="History">History</h2></div>

<div role="note" class="hatnote navigation-not-searchable">See also: <a href="Mercer's_theorem" title="Mercer's theorem">Mercer's theorem</a></div>

Positive-definite kernels, as defined in (1.1), appeared first in 1909 in a paper on integral equations by James Mercer.<a href="#cite_note-4">[4]</a> Several other authors made use of this concept in the following two decades, but none of them explicitly used kernels <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(x,y)=f(x-y)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)=f(x-y)}</annotation>

</semantics>

</math><img src="./5af0cbc7d2ff199ddc9d2ae37a4fe552a7db1c10.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:18.906ex; height:2.843ex;" alt="{\displaystyle K(x,y)=f(x-y)}" loading="lazy">, i.e. p.d. functions (indeed M. Mathias and <a href="Salomon_Bochner" title="Salomon Bochner">S. Bochner</a> seem not to have been aware of the study of p.d. kernels). Mercer’s work arose from Hilbert’s paper of 1904 <a href="#cite_note-5">[5]</a> on <a href="Fredholm_integral_equation" title="Fredholm integral equation">Fredholm integral equations</a> of the second kind:

</mrow>

</mrow>

</msubsup>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f(s)=\varphi (s)-\lambda \int _{a}^{b}K(s,t)\varphi (t)\ \mathrm {d} t.}</annotation>

</semantics>

</math></td> <td></td> <td class="nowrap">1.2</td></tr></tbody></table>

In particular, Hilbert had shown that

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

<mfrac>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

<msup>

<mrow>

<mrow>

</mrow>

</mrow>

</msubsup>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \int _{a}^{b}\int _{a}^{b}K(s,t)x(s)x(t)\ \mathrm {d} s\,\mathrm {d} t=\sum {\frac {1}{\lambda _{n}}}\left[\int _{a}^{b}\psi _{n}(s)x(s)\,\mathrm {d} s\right]^{2},}</annotation>

</semantics>

</math></td> <td></td> <td class="nowrap">1.3</td></tr></tbody></table>

where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy"> is a continuous real symmetric kernel, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x}</annotation>

</semantics>

</math><img src="./87f9e315fd7e2ba406057a97300593c4802b53e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle x}" loading="lazy"> is continuous, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \{\psi _{n}\}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \{\psi _{n}\}}</annotation>

</semantics>

</math><img src="./e889c99c882e72863f4c9608efc4518f448fd514.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.057ex; height:2.843ex;" alt="{\displaystyle \{\psi _{n}\}}" loading="lazy"> is a complete system of <a href="Eigenfunction" title="Eigenfunction">orthonormal eigenfunctions</a>, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lambda _{n}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \lambda _{n}}</annotation>

</semantics>

</math><img src="./093ee22c3daf31b92ff5fa04ba0ce7862283e90c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.574ex; height:2.509ex;" alt="{\displaystyle \lambda _{n}}" loading="lazy">’s are the corresponding <a href="Eigenvalues_and_eigenvectors" title="Eigenvalues and eigenvectors">eigenvalues</a> of (1.2). Hilbert defined a “definite” kernel as one for which the double integral

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

</msubsup>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle J(x)=\int _{a}^{b}\int _{a}^{b}K(s,t)x(s)x(t)\ \mathrm {d} s\;\mathrm {d} t}</annotation>

</semantics>

</math>

satisfies <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle J(x)>0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle J(x)>0}</annotation>

</semantics>

</math><img src="./7a466d6af812a91a3adfc920adb9fff48e68c817.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.871ex; height:2.843ex;" alt="{\displaystyle J(x)>0}" loading="lazy"> except for <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x(t)=0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x(t)=0}</annotation>

</semantics>

</math><img src="./39d5a44241e71c6e25141d278126efa41a5de3dc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.24ex; height:2.843ex;" alt="{\displaystyle x(t)=0}" loading="lazy">. The original object of Mercer’s paper was to characterize the kernels which are definite in the sense of Hilbert, but Mercer soon found that the class of such functions was too restrictive to characterize in terms of determinants. He therefore defined a continuous real symmetric kernel <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(s,t)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(s,t)}</annotation>

</semantics>

</math><img src="./9e3dd971fcd4f23d0690d6f46ecc5d07a086c43d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:6.839ex; height:2.843ex;" alt="{\displaystyle K(s,t)}" loading="lazy"> to be of positive type (i.e. positive-definite) if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle J(x)\geq 0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle J(x)\geq 0}</annotation>

</semantics>

</math><img src="./ca987487cecb4a76d5909a5806849deadccad2e9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.871ex; height:2.843ex;" alt="{\displaystyle J(x)\geq 0}" loading="lazy"> for all real continuous functions <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x}</annotation>

</semantics>

</math><img src="./87f9e315fd7e2ba406057a97300593c4802b53e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle x}" loading="lazy"> on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle [a,b]}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle [a,b]}</annotation>

</semantics>

</math><img src="./9c4b788fc5c637e26ee98b45f89a5c08c85f7935.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.555ex; height:2.843ex;" alt="{\displaystyle [a,b]}" loading="lazy">, and he proved that (1.1) is a necessary and sufficient condition for a kernel to be of positive type. Mercer then proved that for any continuous p.d. kernel the expansion

</mrow>

</munder>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<msub>

</mrow>

</msub>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(s,t)=\sum _{n}{\frac {\psi _{n}(s)\psi _{n}(t)}{\lambda _{n}}}}</annotation>

</semantics>

</math>

holds absolutely and uniformly.

At about the same time W. H. Young,<a href="#cite_note-6">[6]</a> motivated by a different question in the theory of integral equations, showed that for continuous kernels condition (1.1) is equivalent to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle J(x)\geq 0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle J(x)\geq 0}</annotation>

</semantics>

</math><img src="./ca987487cecb4a76d5909a5806849deadccad2e9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.871ex; height:2.843ex;" alt="{\displaystyle J(x)\geq 0}" loading="lazy"> for all <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x\in L^{1}[a,b]}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x\in L^{1}[a,b]}</annotation>

</semantics>

</math><img src="./f44851c00e21916a15d5154187b8c344e47c84ae.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:11.362ex; height:3.176ex;" alt="{\displaystyle x\in L^{1}[a,b]}" loading="lazy">.

E.H. Moore <a href="#cite_note-7">[7]</a><a href="#cite_note-8">[8]</a> initiated the study of a very general kind of p.d. kernel. If <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle E}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle E}</annotation>

</semantics>

</math><img src="./4232c9de2ee3eec0a9c0a19b15ab92daa6223f9b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.776ex; height:2.176ex;" alt="{\displaystyle E}" loading="lazy"> is an abstract set, he calls functions <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(x,y)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)}</annotation>

</semantics>

</math><img src="./6b4554c1b79ecaae2584d99cc3959f2eed7e6d41.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.394ex; height:2.843ex;" alt="{\displaystyle K(x,y)}" loading="lazy"> defined on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle E\times E}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle E\times E}</annotation>

</semantics>

</math><img src="./438347ffb26c796eaac13d2e0cceb8a6a1ad1598.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.392ex; height:2.176ex;" alt="{\displaystyle E\times E}" loading="lazy"> “positive Hermitian matrices” if they satisfy (1.1) for all <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{i}\in E}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x_{i}\in E}</annotation>

</semantics>

</math><img src="./68010a235dd5e6cfcc6313f21918fc4f6348b3f7.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:6.746ex; height:2.509ex;" alt="{\displaystyle x_{i}\in E}" loading="lazy">. Moore was interested in generalization of integral equations and showed that to each such <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy"> there is a Hilbert space <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H}</annotation>

</semantics>

</math><img src="./75a9edddcca2f782014371f75dca39d7e13a9c1b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.064ex; height:2.176ex;" alt="{\displaystyle H}" loading="lazy"> of functions such that, for each <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f\in H,f(y)=(f,K(\cdot ,y))_{H}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f\in H,f(y)=(f,K(\cdot ,y))_{H}}</annotation>

</semantics>

</math><img src="./e344bb0e0bafc63bd0945a9a63c47928760dc2da.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:27.083ex; height:2.843ex;" alt="{\displaystyle f\in H,f(y)=(f,K(\cdot ,y))_{H}}" loading="lazy">. This property is called the reproducing property of the kernel and turns out to have importance in the solution of boundary-value problems for elliptic partial differential equations.

Another line of development in which p.d. kernels played a large role was the theory of harmonics on homogeneous spaces as begun by <a href="%C3%89lie_Cartan" title="Élie Cartan">E. Cartan</a> in 1929, and continued by <a href="Hermann_Weyl" title="Hermann Weyl">H. Weyl</a> and S. Ito. The most comprehensive theory of p.d. kernels in homogeneous spaces is that of <a href="Mark_Krein" title="Mark Krein">M. Krein</a><a href="#cite_note-9">[9]</a> which includes as special cases the work on p.d. functions and irreducible <a href="Positive-definite_function_on_a_group" title="Positive-definite function on a group">unitary representations</a> of locally compact groups.

In probability theory, p.d. kernels arise as covariance kernels of stochastic processes.<a href="#cite_note-10">[10]</a>

<div class="mw-heading mw-heading2"><h2 id="Connection_with_reproducing_kernel_Hilbert_spaces_and_feature_maps">Connection with reproducing kernel Hilbert spaces and feature maps</h2></div>

<div role="note" class="hatnote navigation-not-searchable">Further information: <a href="Reproducing_kernel_Hilbert_space" title="Reproducing kernel Hilbert space">Reproducing kernel Hilbert space</a></div>

Positive-definite kernels provide a framework that encompasses some basic Hilbert space constructions. In the following we present a tight relationship between positive-definite kernels and two mathematical objects, namely reproducing Hilbert spaces and feature maps.

Let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X}</annotation>

</semantics>

</math><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy"> be a set, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H}</annotation>

</semantics>

</math><img src="./75a9edddcca2f782014371f75dca39d7e13a9c1b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.064ex; height:2.176ex;" alt="{\displaystyle H}" loading="lazy"> a Hilbert space of functions <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f:X\to \mathbb {R} }">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f:X\to \mathbb {R} }</annotation>

</semantics>

</math><img src="./669fa4832da4b0b229d77eadb270e95188f2eb10.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:10.488ex; height:2.509ex;" alt="{\displaystyle f:X\to \mathbb {R} }" loading="lazy">, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (\cdot ,\cdot )_{H}:H\times H\to \mathbb {R} }">

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (\cdot ,\cdot )_{H}:H\times H\to \mathbb {R} }</annotation>

</semantics>

</math><img src="./c8a0fc86598149cb0de1720b57aa8aedf5cca412.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:20.025ex; height:2.843ex;" alt="{\displaystyle (\cdot ,\cdot )_{H}:H\times H\to \mathbb {R} }" loading="lazy"> the corresponding inner product on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H}</annotation>

</semantics>

</math><img src="./75a9edddcca2f782014371f75dca39d7e13a9c1b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.064ex; height:2.176ex;" alt="{\displaystyle H}" loading="lazy">. For any <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x\in X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x\in X}</annotation>

</semantics>

</math><img src="./3e580967f68f36743e894aa7944f032dda6ea01d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.15ex; height:2.176ex;" alt="{\displaystyle x\in X}" loading="lazy"> the evaluation functional <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle e_{x}:H\to \mathbb {R} }">

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle e_{x}:H\to \mathbb {R} }</annotation>

</semantics>

</math><img src="./cde61e73f8b0672e7b2ea1d6d07c9cf56ee5468f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:11.549ex; height:2.509ex;" alt="{\displaystyle e_{x}:H\to \mathbb {R} }" loading="lazy"> is defined by <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f\mapsto e_{x}(f)=f(x)}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f\mapsto e_{x}(f)=f(x)}</annotation>

</semantics>

</math><img src="./70b394cb4497271a48c11a37deb681041c6d7812.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:17.753ex; height:2.843ex;" alt="{\displaystyle f\mapsto e_{x}(f)=f(x)}" loading="lazy">.

We first define a reproducing kernel Hilbert space (RKHS):

<blockquote>Definition: Space <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H}</annotation>

</semantics>

</math><img src="./75a9edddcca2f782014371f75dca39d7e13a9c1b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.064ex; height:2.176ex;" alt="{\displaystyle H}" loading="lazy"> is called a reproducing kernel Hilbert space if the evaluation functionals are continuous. </blockquote>

Every RKHS has a special function associated to it, namely the reproducing kernel:

<blockquote>Definition: Reproducing kernel is a function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K:X\times X\to \mathbb {R} }">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K:X\times X\to \mathbb {R} }</annotation>

</semantics>

</math><img src="./61fbac66cd9fa7f18da5fdff4809acca8538a62f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:16.096ex; height:2.176ex;" alt="{\displaystyle K:X\times X\to \mathbb {R} }" loading="lazy"> such that

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K_{x}(\cdot )\in H,\forall x\in X}</annotation>

</semantics>

</math><img src="./51b3a76ae970738d826706f0810a550c7c189ab0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:18.983ex; height:2.843ex;" alt="{\displaystyle K_{x}(\cdot )\in H,\forall x\in X}" loading="lazy">, and</li>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (f,K_{x})_{H}=f(x)}</annotation>

</semantics>

</math><img src="./f700691981c8ec2ad4508a6530c39c555d20f50f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:16.475ex; height:2.843ex;" alt="{\displaystyle (f,K_{x})_{H}=f(x)}" loading="lazy">, for all <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f\in H}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f\in H}</annotation>

</semantics>

</math><img src="./3c9a7c37836068f664ac5e74e37ae36369e86348.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:6.183ex; height:2.509ex;" alt="{\displaystyle f\in H}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x\in X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x\in X}</annotation>

</semantics>

</math><img src="./3e580967f68f36743e894aa7944f032dda6ea01d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.15ex; height:2.176ex;" alt="{\displaystyle x\in X}" loading="lazy">.</li></ol>

The latter property is called the reproducing property. </blockquote>

The following result shows equivalence between RKHS and reproducing kernels:

/* start https://en.wikipedia.org/ */

.mw-parser-output .math_theorem{margin:1em 2em;padding:0.5em 1em 0.4em;border:1px solid #aaa;overflow:hidden}@media(max-width:500px){.mw-parser-output .math_theorem{margin:1em 0em;padding:0.5em 0.5em 0.4em}}

/* end https://en.wikipedia.org/ */

</style><div class="math_theorem" style="">

Theorem— Every reproducing kernel <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy"> induces a unique RKHS, and every RKHS has a unique reproducing kernel.

</div>

Now the connection between positive definite kernels and RKHS is given by the following theorem

Theorem— Every reproducing kernel is positive-definite, and every positive definite kernel defines a unique RKHS, of which it is the unique reproducing kernel.

</div>

Thus, given a positive-definite kernel <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy">, it is possible to build an associated RKHS with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy"> as a reproducing kernel.

As stated earlier, positive definite kernels can be constructed from inner products. This fact can be used to connect p.d. kernels with another interesting object that arises in machine learning applications, namely the feature map. Let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle F}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle F}</annotation>

</semantics>

</math><img src="./545fd099af8541605f7ee55f08225526be88ce57.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.741ex; height:2.176ex;" alt="{\displaystyle F}" loading="lazy"> be a Hilbert space, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (\cdot ,\cdot )_{F}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (\cdot ,\cdot )_{F}}</annotation>

</semantics>

</math><img src="./bd475e20c6eb7025e364737722753d14b659c5e1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.6ex; height:2.843ex;" alt="{\displaystyle (\cdot ,\cdot )_{F}}" loading="lazy"> the corresponding inner product. Any map <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Phi :X\to F}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Phi :X\to F}</annotation>

</semantics>

</math><img src="./67b65ec6a5c3b73a64eb0ea24a0644201752dc81.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:10.95ex; height:2.176ex;" alt="{\displaystyle \Phi :X\to F}" loading="lazy"> is called a feature map. In this case we call <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle F}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle F}</annotation>

</semantics>

</math><img src="./545fd099af8541605f7ee55f08225526be88ce57.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.741ex; height:2.176ex;" alt="{\displaystyle F}" loading="lazy"> the feature space. It is easy to see <a href="#cite_note-11">[11]</a> that every feature map defines a unique p.d. kernel by

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)=(\Phi (x),\Phi (y))_{F}.}</annotation>

</semantics>

</math>

Indeed, positive definiteness of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy"> follows from the p.d. property of the inner product. On the other hand, every p.d. kernel, and its corresponding RKHS, have many associated feature maps. For example: Let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle F=H}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle F=H}</annotation>

</semantics>

</math><img src="./fe446b443ea73187109cd61ef7236a2bfe75ddc5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.903ex; height:2.176ex;" alt="{\displaystyle F=H}" loading="lazy">, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Phi (x)=K_{x}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Phi (x)=K_{x}}</annotation>

</semantics>

</math><img src="./0bc90b376d53b8cf9837555d14797a7b1e0623b4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:11.061ex; height:2.843ex;" alt="{\displaystyle \Phi (x)=K_{x}}" loading="lazy"> for all <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x\in X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x\in X}</annotation>

</semantics>

</math><img src="./3e580967f68f36743e894aa7944f032dda6ea01d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.15ex; height:2.176ex;" alt="{\displaystyle x\in X}" loading="lazy">. Then <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (\Phi (x),\Phi (y))_{F}=(K_{x},K_{y})_{H}=K(x,y)}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (\Phi (x),\Phi (y))_{F}=(K_{x},K_{y})_{H}=K(x,y)}</annotation>

</semantics>

</math><img src="./6011d1523fa4d227f9e06178256da4035f4b6bbb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:38.06ex; height:3.009ex;" alt="{\displaystyle (\Phi (x),\Phi (y))_{F}=(K_{x},K_{y})_{H}=K(x,y)}" loading="lazy">, by the reproducing property.

This suggests a new look at p.d. kernels as inner products in appropriate Hilbert spaces, or in other words p.d. kernels can be viewed as similarity maps which quantify effectively how similar two points <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x}</annotation>

</semantics>

</math><img src="./87f9e315fd7e2ba406057a97300593c4802b53e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle x}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle y}</annotation>

</semantics>

</math><img src="./b8a6208ec717213d4317e666f1ae872e00620a0d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.155ex; height:2.009ex;" alt="{\displaystyle y}" loading="lazy"> are through the value <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(x,y)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)}</annotation>

</semantics>

</math><img src="./6b4554c1b79ecaae2584d99cc3959f2eed7e6d41.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.394ex; height:2.843ex;" alt="{\displaystyle K(x,y)}" loading="lazy">. Moreover, through the equivalence of p.d. kernels and its corresponding RKHS, every feature map can be used to construct a RKHS.

<div class="mw-heading mw-heading2"><h2 id="Kernels_and_distances">Kernels and distances</h2></div>

Kernel methods are often compared to distance based methods such as <a href="K-nearest_neighbors_algorithm" title="K-nearest neighbors algorithm">nearest neighbors</a>. In this section we discuss parallels between their two respective ingredients, namely kernels <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy"> and distances <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d}</annotation>

</semantics>

</math><img src="./e85ff03cbe0c7341af6b982e47e9f90d235c66ab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.216ex; height:2.176ex;" alt="{\displaystyle d}" loading="lazy">.

Here by a distance function between each pair of elements of some set <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X}</annotation>

</semantics>

</math><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy">, we mean a <a href="Metric_(mathematics)" class="mw-redirect" title="Metric (mathematics)">metric</a> defined on that set, i.e. any nonnegative-valued function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d}</annotation>

</semantics>

</math><img src="./e85ff03cbe0c7341af6b982e47e9f90d235c66ab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.216ex; height:2.176ex;" alt="{\displaystyle d}" loading="lazy"> on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {X}}\times {\mathcal {X}}}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}\times {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./d1fe1af80b732982dbeecc8f0199bceab1a9dd04.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.591ex; height:2.176ex;" alt="{\displaystyle {\mathcal {X}}\times {\mathcal {X}}}" loading="lazy"> which satisfies

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d(x,y)\geq 0}</annotation>

</semantics>

</math><img src="./1309cec878e6d9effccd8a2d2ed065a8e82bfa82.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:10.805ex; height:2.843ex;" alt="{\displaystyle d(x,y)\geq 0}" loading="lazy">, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d(x,y)=0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d(x,y)=0}</annotation>

</semantics>

</math><img src="./3329b396b395dedb4a470dea0aacea4b485070a8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:10.805ex; height:2.843ex;" alt="{\displaystyle d(x,y)=0}" loading="lazy"> if and only if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x=y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x=y}</annotation>

</semantics>

</math><img src="./409a91214d63eabe46ec10ff3cbba689ab687366.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:5.584ex; height:2.009ex;" alt="{\displaystyle x=y}" loading="lazy">,</li>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d(x,y)=d(y,x),}</annotation>

</semantics>

</math><img src="./c2087dc06278d4cfc025ac6c21f3118028ab6c78.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:16.834ex; height:2.843ex;" alt="{\displaystyle d(x,y)=d(y,x),}" loading="lazy"></li>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d(x,z)\leq d(x,y)+d(y,z).}</annotation>

</semantics>

</math><img src="./1e7072502a79725a25151d5c96b7310e7dbb97a3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:25.91ex; height:2.843ex;" alt="{\displaystyle d(x,z)\leq d(x,y)+d(y,z).}" loading="lazy"></li></ul>

One link between distances and p.d. kernels is given by a particular kind of kernel, called a negative definite kernel, and defined as follows

<blockquote>Definition: A symmetric function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \psi :{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {R} }">

</mrow>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \psi :{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {R} }</annotation>

</semantics>

</math><img src="./7e9a0c8f9186f504075e470a9875994107ae9b6b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:15.334ex; height:2.509ex;" alt="{\displaystyle \psi :{\mathcal {X}}\times {\mathcal {X}}\to \mathbb {R} }" loading="lazy"> is called a negative definite (n.d.) kernel on <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {X}}}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./8c7e5461c5286852df4ef652fca7e4b0b63030e9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.875ex; height:2.176ex;" alt="{\displaystyle {\mathcal {X}}}" loading="lazy"> if

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sum _{i,j=1}^{n}c_{i}c_{j}\psi (x_{i},x_{j})\leq 0}</annotation>

</semantics>

</math></td> <td></td> <td class="nowrap">1.4</td></tr></tbody></table>

holds for any <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\in \mathbb {N} ,x_{1},\dots ,x_{n}\in {\mathcal {X}},}">

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\in \mathbb {N} ,x_{1},\dots ,x_{n}\in {\mathcal {X}},}</annotation>

</semantics>

</math><img src="./4d4a775770a9330d3775ba96e567091dca09d6b8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:22.421ex; height:2.509ex;" alt="{\displaystyle n\in \mathbb {N} ,x_{1},\dots ,x_{n}\in {\mathcal {X}},}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle c_{1},\dots ,c_{n}\in \mathbb {R} }">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle c_{1},\dots ,c_{n}\in \mathbb {R} }</annotation>

</semantics>

</math><img src="./351f53aec2584e0b1d6a0b157f7a5bd855f0f4d9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:13.983ex; height:2.509ex;" alt="{\displaystyle c_{1},\dots ,c_{n}\in \mathbb {R} }" loading="lazy"> such that <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\textstyle \sum _{i=1}^{n}c_{i}=0}">

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\textstyle \sum _{i=1}^{n}c_{i}=0}</annotation>

</semantics>

</math><img src="./2a40463dc80df707b33a8e941bff68b8698b78d0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:11.809ex; height:3.176ex;" alt="{\textstyle \sum _{i=1}^{n}c_{i}=0}" loading="lazy">.

</blockquote>

The parallel between n.d. kernels and distances is in the following: whenever a n.d. kernel vanishes on the set <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \{(x,x):x\in {\mathcal {X}}\}}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \{(x,x):x\in {\mathcal {X}}\}}</annotation>

</semantics>

</math><img src="./66db1f2c85fda30ce5e3c6b1bcb6a35515c9b10a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:15.811ex; height:2.843ex;" alt="{\displaystyle \{(x,x):x\in {\mathcal {X}}\}}" loading="lazy">, and is zero only on this set, then its square root is a distance for <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {X}}}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./8c7e5461c5286852df4ef652fca7e4b0b63030e9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.875ex; height:2.176ex;" alt="{\displaystyle {\mathcal {X}}}" loading="lazy">.<a href="#cite_note-12">[12]</a> At the same time each distance does not correspond necessarily to a n.d. kernel. This is only true for Hilbertian distances, where distance <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d}</annotation>

</semantics>

</math><img src="./e85ff03cbe0c7341af6b982e47e9f90d235c66ab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.216ex; height:2.176ex;" alt="{\displaystyle d}" loading="lazy"> is called Hilbertian if one can embed the metric space <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle ({\mathcal {X}},d)}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle ({\mathcal {X}},d)}</annotation>

</semantics>

</math><img src="./cfd6af64491d693bbcd2e0b0b236cad90e54cc47.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.935ex; height:2.843ex;" alt="{\displaystyle ({\mathcal {X}},d)}" loading="lazy"> <a href="Isometry" title="Isometry">isometrically</a> into some Hilbert space.

On the other hand, n.d. kernels can be identified with a subfamily of p.d. kernels known as infinitely divisible kernels. A nonnegative-valued kernel <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy"> is said to be infinitely divisible if for every <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\in \mathbb {N} }">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\in \mathbb {N} }</annotation>

</semantics>

</math><img src="./d059936e77a2d707e9ee0a1d9575a1d693ce5d0b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:5.913ex; height:2.176ex;" alt="{\displaystyle n\in \mathbb {N} }" loading="lazy"> there exists a positive-definite kernel <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K_{n}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K_{n}}</annotation>

</semantics>

</math><img src="./ea2b988ea630d2c5571afe47efa3d3b251708acb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.192ex; height:2.509ex;" alt="{\displaystyle K_{n}}" loading="lazy"> such that <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K=(K_{n})^{n}}">

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K=(K_{n})^{n}}</annotation>

</semantics>

</math><img src="./80584c65e3a83c3457e27715f9c795ca3aee8245.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:11.384ex; height:2.843ex;" alt="{\displaystyle K=(K_{n})^{n}}" loading="lazy">.

Another link is that a p.d. kernel induces a <a href="Pseudometric_space" title="Pseudometric space">pseudometric</a>, where the first constraint on the distance function is loosened to allow <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d(x,y)=0}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d(x,y)=0}</annotation>

</semantics>

</math><img src="./3329b396b395dedb4a470dea0aacea4b485070a8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:10.805ex; height:2.843ex;" alt="{\displaystyle d(x,y)=0}" loading="lazy"> for <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x\neq y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x\neq y}</annotation>

</semantics>

</math><img src="./f51b711ca7f932963cdb268b0817dc72d6258733.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.584ex; height:2.676ex;" alt="{\displaystyle x\neq y}" loading="lazy">. Given a positive-definite kernel <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy">, we can define a distance function as:

<msqrt>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d(x,y)={\sqrt {K(x,x)-2K(x,y)+K(y,y)}}}</annotation>

</semantics>

</math>

<div class="mw-heading mw-heading2"><h2 id="Some_applications">Some applications</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Kernels_in_machine_learning">Kernels in machine learning</h3></div>

<div role="note" class="hatnote navigation-not-searchable">Further information: <a href="Kernel_method" title="Kernel method">Kernel method</a></div>

Positive-definite kernels, through their equivalence with reproducing kernel Hilbert spaces (RKHS), are particularly important in the field of <a href="Statistical_learning_theory" title="Statistical learning theory">statistical learning theory</a> because of the celebrated <a href="Representer_theorem" title="Representer theorem">representer theorem</a> which states that every minimizer function in an RKHS can be written as a linear combination of the kernel function evaluated at the training points. This is a practically useful result as it effectively simplifies the empirical risk minimization problem from an infinite dimensional to a finite dimensional optimization problem.

<div class="mw-heading mw-heading3"><h3 id="Kernels_in_probabilistic_models">Kernels in probabilistic models</h3></div>

There are several different ways in which kernels arise in probability theory.

<ul><li>Nondeterministic recovery problems: Assume that we want to find the response <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f(x)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f(x)}</annotation>

</semantics>

</math><img src="./202945cce41ecebb6f643f31d119c514bec7a074.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.418ex; height:2.843ex;" alt="{\displaystyle f(x)}" loading="lazy"> of an unknown model function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> at a new point <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x}</annotation>

</semantics>

</math><img src="./87f9e315fd7e2ba406057a97300593c4802b53e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle x}" loading="lazy"> of a set <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {X}}}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./8c7e5461c5286852df4ef652fca7e4b0b63030e9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.875ex; height:2.176ex;" alt="{\displaystyle {\mathcal {X}}}" loading="lazy">, provided that we have a sample of input-response pairs <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle (x_{i},f_{i})=(x_{i},f(x_{i}))}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle (x_{i},f_{i})=(x_{i},f(x_{i}))}</annotation>

</semantics>

</math><img src="./be7a5cc1c46b5f67a516134e75c13fa3b0c81819.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:20.2ex; height:2.843ex;" alt="{\displaystyle (x_{i},f_{i})=(x_{i},f(x_{i}))}" loading="lazy"> given by observation or experiment. The response <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f_{i}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{i}}</annotation>

</semantics>

</math><img src="./65da883ca3d16b461e46c94777b0d9c4aa010e79.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.939ex; height:2.509ex;" alt="{\displaystyle f_{i}}" loading="lazy"> at <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{i}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x_{i}}</annotation>

</semantics>

</math><img src="./e87000dd6142b81d041896a30fe58f0c3acb2158.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.129ex; height:2.009ex;" alt="{\displaystyle x_{i}}" loading="lazy"> is not a fixed function of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{i}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x_{i}}</annotation>

</semantics>

</math><img src="./e87000dd6142b81d041896a30fe58f0c3acb2158.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.129ex; height:2.009ex;" alt="{\displaystyle x_{i}}" loading="lazy"> but rather a realization of a real-valued random variable <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z(x_{i})}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z(x_{i})}</annotation>

</semantics>

</math><img src="./05f948fed095112009f8e926efcc7dbd0655ccf9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.619ex; height:2.843ex;" alt="{\displaystyle Z(x_{i})}" loading="lazy">. The goal is to get information about the function <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle E[Z(x_{i})]}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle E[Z(x_{i})]}</annotation>

</semantics>

</math><img src="./22f2db6135181a0e1c58a477d3db36e92904b69d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.688ex; height:2.843ex;" alt="{\displaystyle E[Z(x_{i})]}" loading="lazy"> which replaces <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> in the deterministic setting. For two elements <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x,y\in {\mathcal {X}}}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x,y\in {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./77414a3f1390a5372a319c960e72b098d782034b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:8.235ex; height:2.509ex;" alt="{\displaystyle x,y\in {\mathcal {X}}}" loading="lazy"> the random variables <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z(x)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z(x)}</annotation>

</semantics>

</math><img src="./6c2ffb6f5d6e9eeab2867ef3c37ea13f9c294ac8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.819ex; height:2.843ex;" alt="{\displaystyle Z(x)}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z(y)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z(y)}</annotation>

</semantics>

</math><img src="./1bc4476d592d9d9e8b014ab90369f4597c7dce52.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.645ex; height:2.843ex;" alt="{\displaystyle Z(y)}" loading="lazy"> will not be uncorrelated, because if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x}</annotation>

</semantics>

</math><img src="./87f9e315fd7e2ba406057a97300593c4802b53e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle x}" loading="lazy"> is too close to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle y}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle y}</annotation>

</semantics>

</math><img src="./b8a6208ec717213d4317e666f1ae872e00620a0d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.155ex; height:2.009ex;" alt="{\displaystyle y}" loading="lazy"> the random experiments described by <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z(x)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z(x)}</annotation>

</semantics>

</math><img src="./6c2ffb6f5d6e9eeab2867ef3c37ea13f9c294ac8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.819ex; height:2.843ex;" alt="{\displaystyle Z(x)}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z(y)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z(y)}</annotation>

</semantics>

</math><img src="./1bc4476d592d9d9e8b014ab90369f4597c7dce52.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.645ex; height:2.843ex;" alt="{\displaystyle Z(y)}" loading="lazy"> will often show similar behaviour. This is described by a covariance kernel <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(x,y)=E[Z(x)\cdot Z(y)]}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)=E[Z(x)\cdot Z(y)]}</annotation>

</semantics>

</math><img src="./1c0a3c96ebb227a2e9110f93c1e01d6fc998b334.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:24.706ex; height:2.843ex;" alt="{\displaystyle K(x,y)=E[Z(x)\cdot Z(y)]}" loading="lazy">. Such a kernel exists and is positive-definite under weak additional assumptions. Now a good estimate for <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z(x)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z(x)}</annotation>

</semantics>

</math><img src="./6c2ffb6f5d6e9eeab2867ef3c37ea13f9c294ac8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.819ex; height:2.843ex;" alt="{\displaystyle Z(x)}" loading="lazy"> can be obtained by using kernel interpolation with the covariance kernel, ignoring the probabilistic background completely.</li></ul>

Assume now that a noise variable <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \epsilon (x)}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \epsilon (x)}</annotation>

</semantics>

</math><img src="./486bd3944e70fc279f235f7db3a20f743a91084e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.083ex; height:2.843ex;" alt="{\displaystyle \epsilon (x)}" loading="lazy">, with zero mean and variance <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \sigma ^{2}}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \sigma ^{2}}</annotation>

</semantics>

</math><img src="./53a5c55e536acf250c1d3e0f754be5692b843ef5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.385ex; height:2.676ex;" alt="{\displaystyle \sigma ^{2}}" loading="lazy">, is added to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x}</annotation>

</semantics>

</math><img src="./87f9e315fd7e2ba406057a97300593c4802b53e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle x}" loading="lazy">, such that the noise is independent for different <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x}</annotation>

</semantics>

</math><img src="./87f9e315fd7e2ba406057a97300593c4802b53e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.33ex; height:1.676ex;" alt="{\displaystyle x}" loading="lazy"> and independent of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Z}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Z}</annotation>

</semantics>

</math><img src="./1cc6b75e09a8aa3f04d8584b11db534f88fb56bd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.68ex; height:2.176ex;" alt="{\displaystyle Z}" loading="lazy"> there, then the problem of finding a good estimate for <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> is identical to the above one, but with a modified kernel given by <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(x,y)=E[Z(x)\cdot Z(y)]+\sigma ^{2}\delta _{xy}}">

<msup>

</mrow>

</msup>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(x,y)=E[Z(x)\cdot Z(y)]+\sigma ^{2}\delta _{xy}}</annotation>

</semantics>

</math><img src="./73d07bdfe784687f03aa2e4068c345e048c2cbd5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:32.953ex; height:3.343ex;" alt="{\displaystyle K(x,y)=E[Z(x)\cdot Z(y)]+\sigma ^{2}\delta _{xy}}" loading="lazy">.

<ul><li>Density estimation by kernels: The problem is to recover the density <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f}</annotation>

</semantics>

</math><img src="./132e57acb643253e7810ee9702d9581f159a1c61.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.279ex; height:2.509ex;" alt="{\displaystyle f}" loading="lazy"> of a multivariate distribution over a domain <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mathcal {X}}}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./8c7e5461c5286852df4ef652fca7e4b0b63030e9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.875ex; height:2.176ex;" alt="{\displaystyle {\mathcal {X}}}" loading="lazy">, from a large sample <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}</annotation>

</semantics>

</math><img src="./17bb9dc93c447ce33bb7fbcbc2e147823ce03dc0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:14.826ex; height:2.509ex;" alt="{\displaystyle x_{1},\dots ,x_{n}\in {\mathcal {X}}}" loading="lazy"> including repetitions. Where sampling points lie dense, the true density function must take large values. A simple density estimate is possible by counting the number of samples in each cell of a grid, and plotting the resulting histogram, which yields a piecewise constant density estimate. A better estimate can be obtained by using a nonnegative translation invariant kernel <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K}</annotation>

</semantics>

</math><img src="./2b76fce82a62ed5461908f0dc8f037de4e3686b0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:2.066ex; height:2.176ex;" alt="{\displaystyle K}" loading="lazy">, with total integral equal to one, and define <math display="block" xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f(x)={\frac {1}{n}}\sum _{i=1}^{n}K\left({\frac {x-x_{i}}{h}}\right)}">

<mfrac>

</mfrac>

</mrow>

</mrow>

</mrow>

</munderover>

<mrow>

<mfrac>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f(x)={\frac {1}{n}}\sum _{i=1}^{n}K\left({\frac {x-x_{i}}{h}}\right)}</annotation>

</semantics>

</math> as a smooth estimate.</li></ul>

<div class="mw-heading mw-heading3"><h3 id="Numerical_solution_of_partial_differential_equations">Numerical solution of partial differential equations</h3></div>

<div role="note" class="hatnote navigation-not-searchable">Further information: <a href="Meshfree_methods" title="Meshfree methods">Meshfree methods</a></div>

One of the greatest application areas of so-called <a href="Meshfree_methods" title="Meshfree methods">meshfree methods</a> is in the numerical solution of <a href="Partial_differential_equation" title="Partial differential equation">PDEs</a>. Some of the popular meshfree methods are closely related to positive-definite kernels (such as meshless local Petrov Galerkin (MLPG), <a href="Reproducing_kernel_particle_method" class="mw-redirect" title="Reproducing kernel particle method">Reproducing kernel particle method (RKPM)</a> and <a href="Smoothed-particle_hydrodynamics" title="Smoothed-particle hydrodynamics">smoothed-particle hydrodynamics (SPH)</a>). These methods use radial basis kernel for <a href="Collocation_method" title="Collocation method">collocation</a>.<a href="#cite_note-13">[13]</a>

<div class="mw-heading mw-heading3"><h3 id="Stinespring_dilation_theorem">Stinespring dilation theorem</h3></div>

<div role="note" class="hatnote navigation-not-searchable">Further information: <a href="Stinespring_dilation_theorem" title="Stinespring dilation theorem">Stinespring dilation theorem</a></div>

<div class="mw-heading mw-heading3"><h3 id="Other_applications">Other applications</h3></div>

In the literature on computer experiments <a href="#cite_note-14">[14]</a> and other engineering experiments, one increasingly encounters models based on p.d. kernels, RBFs or <a href="Kriging" title="Kriging">kriging</a>. One such topic is <a href="Response_surface_methodology" title="Response surface methodology">response surface methodology</a>. Other types of applications that boil down to data fitting are <a href="Rapid_prototyping" title="Rapid prototyping">rapid prototyping</a> and <a href="Computer_graphics_(computer_science)" title="Computer graphics (computer science)">computer graphics</a>. Here one often uses implicit surface models to approximate or interpolate point cloud data.

Applications of p.d. kernels in various other branches of mathematics are in multivariate integration, multivariate optimization, and in numerical analysis and scientific computing, where one studies fast, accurate and adaptive algorithms ideally implemented in high-performance computing environments.<a href="#cite_note-15">[15]</a>

<ul><li><a href="Covariance_function" title="Covariance function">Covariance function</a></li>

<li><a href="Integral_equation" title="Integral equation">Integral equation</a></li>

<li><a href="Integral_transform" title="Integral transform">Integral transform</a></li>

<li><a href="Positive-definite_function_on_a_group" title="Positive-definite function on a group">Positive-definite function on a group</a></li>

<li><a href="Reproducing_kernel_Hilbert_space" title="Reproducing kernel Hilbert space">Reproducing kernel Hilbert space</a></li>

<li><a href="Kernel_method" title="Kernel method">Kernel method</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist reflist-columns references-column-width" style="column-width: 30em;">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFBerezanskij1968" class="citation book cs1">Berezanskij, Jurij Makarovič (1968). Expansions in eigenfunctions of selfadjoint operators. Providence, RI: American Mathematical Soc. pp. 45–47. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-8218-1567-0</bdi>.</cite>

</li>

<li id="cite_note-2"><a href="#cite_ref-2">^</a> Hein, M. and Bousquet, O. (2005). "<a rel="nofollow" class="external text" href="https://pdfs.semanticscholar.org/7822/69ba228ab3c245ea62f8da2fee141a40f486.pdf">Hilbertian metrics and positive definite kernels on probability measures</a>". In Ghahramani, Z. and Cowell, R., editors, Proceedings of AISTATS 2005.

</li>

<li id="cite_note-3"><a href="#cite_ref-3">^</a> Lin, Hsuan-Tien, and Chih-Jen Lin. "A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods." Neural Comput 3.1-32 (2003): 16.

</li>

<li id="cite_note-4"><a href="#cite_ref-4">^</a> Mercer, J. (1909). “Functions of positive and negative type and their connection with the theory of integral equations”. Philosophical Transactions of the Royal Society of London, Series A 209, pp. 415–446.

</li>

<li id="cite_note-5"><a href="#cite_ref-5">^</a> Hilbert, D. (1904). "Grundzuge einer allgemeinen Theorie der linearen Integralgleichungen I", Gott. Nachrichten, math.-phys. K1 (1904), pp. 49–91.

</li>

<li id="cite_note-6"><a href="#cite_ref-6">^</a> Young, W. H. (1909). "A note on a class of symmetric functions and on a theorem required in the theory of integral equations", Philos. Trans. Roy.Soc. London, Ser. A, 209, pp. 415–446.

</li>

<li id="cite_note-7"><a href="#cite_ref-7">^</a> Moore, E.H. (1916). "On properly positive Hermitian matrices", Bull. Amer. Math. Soc. 23, 59, pp. 66–67.

</li>

<li id="cite_note-8"><a href="#cite_ref-8">^</a> Moore, E.H. (1935). "General Analysis, Part I", Memoirs Amer. Philos. Soc. 1, Philadelphia.

</li>

<li id="cite_note-9"><a href="#cite_ref-9">^</a> Krein. M (1949/1950). "Hermitian-positive kernels on homogeneous spaces I and II" (in Russian), Ukrain. Mat. Z. 1(1949), pp. 64–98, and 2(1950), pp. 10–59. English translation: Amer. Math. Soc. Translations Ser. 2, 34 (1963), pp. 69–164.

</li>

<li id="cite_note-10"><a href="#cite_ref-10">^</a> Loève, M. (1960). "Probability theory", 2nd ed., Van Nostrand, Princeton, N.J.

</li>

<li id="cite_note-11"><a href="#cite_ref-11">^</a> Rosasco, L. and Poggio, T. (2015). "A Regularization Tour of Machine Learning – MIT 9.520 Lecture Notes" Manuscript.

</li>

<li id="cite_note-12"><a href="#cite_ref-12">^</a> Berg, C., Christensen, J. P. R., and Ressel, P. (1984). "Harmonic Analysis on Semigroups". Number 100 in Graduate Texts in Mathematics, Springer Verlag.

</li>

<li id="cite_note-13"><a href="#cite_ref-13">^</a> Schaback, R. and Wendland, H. (2006). "Kernel Techniques: From Machine Learning to Meshless Methods", Cambridge University Press, Acta Numerica (2006), pp. 1–97.

</li>

<li id="cite_note-14"><a href="#cite_ref-14">^</a> Haaland, B. and Qian, P. Z. G. (2010). "Accurate emulators for large-scale computer experiments", Ann. Stat.

</li>

<li id="cite_note-15"><a href="#cite_ref-15">^</a> Gumerov, N. A. and Duraiswami, R. (2007). "<a rel="nofollow" class="external text" href="http://users.umiacs.umd.edu/~ramani/pubs/Gumerov_Duraiswami_SISC_29_1876_1899_2007.pdf">Fast radial basis function interpolation via preconditioned Krylov iteration</a>". SIAM J. Scient. Computing 29/5, pp. 1876–1899.

</li>

</ol></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-05-27" href="https://en.wikipedia.org/wiki/?title=Positive-definite_kernel&oldid=1292458062">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>